{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "machine_shape": "hm"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f58Xwgoziq5R"
      },
      "source": [
        "![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/component_examples/sequence2sequence/bart_transformer.ipynb)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Bart Transformer**"
      ],
      "metadata": {
        "id": "9P0scwAErbgg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "BART is a transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. BART is pre-trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.\n",
        "\n",
        "BART is particularly effective when fine-tuned for text generation (e.g. summarization, translation) but also works well for comprehension tasks (e.g. text classification, question answering). This particular checkpoint has been fine-tuned on CNN Daily Mail, a large collection of text-summary pairs."
      ],
      "metadata": {
        "id": "ZAiyR-nNtCxU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**All the available models:**\n",
        "\n",
        "| Language | nlu.load() reference            | Spark NLP Model reference                                                                     |\n",
        "|----------|---------------------------------|-----------------------------------------------------------------------------------------------|\n",
        "| English  | en.seq2seq.distilbart_xsum_12_6 | [distilbart_xsum_12_6](https://sparknlp.org//2023/04/07/distilbart_xsum_12_6_en.html) |\n",
        "| English  | en.seq2seq.bart_large_cnn       | [bart_large_cnn](https://sparknlp.org//2023/04/09/bart_large_cnn_en.html)             |\n",
        "| English  | en.seq2seq.distilbart_cnn_6_6   | [distilbart_cnn_6_6](https://sparknlp.org//2023/04/09/distilbart_cnn_6_6_en.html)     |\n",
        "| English  | en.seq2seq.distilbart_cnn_12_6  | [distilbart_cnn_12_6](https://sparknlp.org//2023/05/09/distilbart_cnn_12_6_en.html)   |\n",
        "| English  | en.seq2seq.distilbart_xsum_6_6  | [distilbart_xsum_6_6](https://sparknlp.org//2023/05/09/distilbart_xsum_6_6_en.html)   |\n"
      ],
      "metadata": {
        "id": "SFCajZiUtIOQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%capture\n",
        "\n",
        "!wget http://setup.johnsnowlabs.com/nlu/colab.sh -O - | bash"
      ],
      "metadata": {
        "id": "SortrsXZsHnw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install pyspark==3.0.1\n",
        "!pip install pandas==1.5.0"
      ],
      "metadata": {
        "id": "-lof6HDGsAc6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 📑  **en.seq2seq.distilbart_cnn_12_6**\n",
        "\n",
        "BART model pre-trained on English language, and fine-tuned on CNN Daily Mail. It was introduced in the paper BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension by Lewis et al. and first released in [this repository (https://github.com/pytorch/fairseq/tree/master/examples/bart)."
      ],
      "metadata": {
        "id": "5vddqMKptPQA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nlu\n",
        "model = nlu.load('en.seq2seq.distilbart_cnn_12_6')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7_wYZ4fVsEd5",
        "outputId": "36aab03c-4498-49d7-d37b-24bf4cf6d2aa"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Warning::Spark Session already created, some configs may not take.\n",
            "Warning::Spark Session already created, some configs may not take.\n",
            "distilbart_cnn_12_6 download started this may take some time.\n",
            "Approximate size to download 830.1 MB\n",
            "[OK!]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model['bart_transformer'].setTask(\"summarize: \")\n",
        "model['bart_transformer'].setMaxOutputLength(200)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2lpLHIUkulIP",
        "outputId": "beb4f5fd-2c9b-4df3-dc9f-12a7ffabae5c"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "BartTRANSFORMER_916a31f5a963"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data = '''LONDON, England (Reuters) -- Harry Potter star Daniel Radcliffe gains access to a reported £20 million ($41.1 million) fortune as he turns 18 on Monday, but he insists the money won't cast a spell on him. Daniel Radcliffe as Harry Potter in \"Harry Potter and the Order of the Phoenix\" To the disappointment of gossip columnists around the world, the young actor says he has no plans to fritter his cash away on fast cars, drink and celebrity parties. \"I don't plan to be one of those people who, as soon as they turn 18, suddenly buy themselves a massive sports car collection or something similar,\" he told an Australian interviewer earlier this month. \"I don't think I'll be particularly extravagant. \"The things I like buying are things that cost about 10 pounds -- books and CDs and DVDs.\" At 18, Radcliffe will be able to gamble in a casino, buy a drink in a pub or see the horror film \"Hostel: Part II,\" currently six places below his number one movie on the UK box office chart. Details of how he'll mark his landmark birthday are under wraps. His agent and publicist had no comment on his plans. \"I'll definitely have some sort of party,\" he said in an interview. \"Hopefully none of you will be reading about it.\" Radcliffe's earnings from the first five Potter films have been held in a trust fund which he has not been able to touch. Despite his growing fame and riches, the actor says he is keeping his feet firmly on the ground. \"People are always looking to say 'kid star goes off the rails,'\" he told reporters last month. \"But I try very hard not to go that way because it would be too easy for them.\" His latest outing as the boy wizard in \"Harry Potter and the Order of the Phoenix\" is breaking records on both sides of the Atlantic and he will reprise the role in the last two films. Watch I-Reporter give her review of Potter's latest » . There is life beyond Potter, however. The Londoner has filmed a TV movie called \"My Boy Jack,\" about author Rudyard Kipling and his son, due for release later this year. He will also appear in \"December Boys,\" an Australian film about four boys who escape an orphanage. Earlier this year, he made his stage debut playing a tortured teenager in Peter Shaffer's \"Equus.\" Meanwhile, he is braced for even closer media scrutiny now that he's legally an adult: \"I just think I'm going to be more sort of fair game,\" he told Reuters. E-mail to a friend . Copyright 2007 Reuters. All rights reserved.This material may not be published, broadcast, rewritten, or redistributed.'''"
      ],
      "metadata": {
        "id": "HNoeU7yeulRf"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df = model.predict(data)"
      ],
      "metadata": {
        "id": "SXlkMUfhulKm"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df.generated.iloc[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "skDqiFYvseHF",
        "outputId": "3ad97e28-1973-455c-8481-fab01a980ff4"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\" Daniel Radcliffe gains access to a reported � 20 million $ 41 . 1 million fortune . Harry Potter star Daniel Radcliffe turns 18 on Monday . Radcliffe insists the money won't cast a spell on him .\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "7drirWvAteUV"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}
